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(54) Real-time image enhancement techniques 

(57) Improved image enhancement methods are 
disclosed which may be performed on conventional vid- 
eo processing systems. These methods operate in real 
time to improve the appearance of images as perceived 
by the human eye, and/or to render these images more 



suitable for computer analysis. A first preferred embod- 
iment enhances color images without degrading the fi- 
delity of the image. A second preferred embodiment en- 
hances scanned gray-level document images by de- 
creasing the blurring of the image and increasing the 
contrast between light and dark regions of the image. 
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Description 

Background Of the Invention 

1. Field of the Invention 

The invention relates generally to image enhance- 
ment techniques, and more specifically to real-time im- 
age enhancement methods which are performed on tex- 
tual and/or color photographic images. 

2. Description of the Prior Art 

Image enhancement is a process by which acquired 
image data values are manipulated. Such manipulation 
is typically directed to improving the appearance of the 
data as perceived by humans, and/or to render the data 
more suitable for machine analysis of the image. Image 
enhancement processes are distinguishable from im- 
age restoration processes in that one objective of res- 
toration processes is to improve the fidelity and/or ac- 
curacy of a reproduced image with respect to the original 
image, whereas enhancement processes place greater 
emphasis on human visual perception and/or the suita- 
bility ol the processed image for computer analysis. 

Various image enhancement processes have been 
developed for improving the perceived clarity, intelligi- 
bility, and/or quality of images. These image enhance- 
ment processes attempt to render images more pleas- 
ing to human eyes, irrespective of whether or not these 
images are accurately reproduced. Oftentimes, a dis- 
torted image is more pleasing to the human eye than a 
perfectly reproduced copy of the original image. For ex- 
ample, if overshoot or undershoot is incorporated into 
the edges of an image, the resulting image is typically 
perceived as being more pleasing than an image without 
such undershoot or overshoot. 

Many prior art image enhancement processes have 
focused on the improvement of gray-level images, but 
a direct extension of these processes into the color do- 
main results in degraded or suboptimal images. At the 
present time, there is a notable lack of image, enhance- 
ment techniques which are optimized for use in conjunc- 
tion with color images. The widespread distribution of 
color images, the growing popularity of multimedia ap- 
plications, the current proliferation of personal comput- 
ing devices, and the development of high -definition tel- 
evision systems all serve to increase the demand for im- 
proved color image enhancement methods. Such image 
enhancement methods should be optimized for use in 
conjunction with color images, and should not merely 
constitute the reapplication of a gray-level method to the 
color domain. Moreover, such image enhancement 
methods should operate in realtime using existing hard- 
ware platforms. 

Although it is possible to enhance a color image by 
applying existing monochrome image enhancement al- 
gorithms to each color component (red, green, and blue) 



individually, the resulting image may not be enhanced 
optimally. Image distortion will result if the red, green, 
and blue components are not properly recombined after 
the monochrome image enhancement algorithms are 
5 applied. Moreover, image distortion will also result if the 
red, green, and blue components are processed inde- 
pendently, without considering correlation among the 
components. 

10 Summary of the Invention 

Improved image enhancement methods are dis- 
closed which may be performed on conventional video 
processing systems. These methods operate in real 
1 * time to improve the appearance of images as perceived 
by the human eye, and/or to render these images more 
suitable for computer analysis. A first preferred embod- 
iment enhances color images without degrading the 
color fidelity of the image. A second preferred embodi- 

20 ment enhances gray-level textual images by decreasing 
the blurriness of the image, by increasing the contrast 
between light and dark regions of the image, and by re- 
moving image artifacts which are introduced by conven- 
tional block-based compression algorithms 

2* According to the first preferred embodiment, a color 
image is represented by a plurality of pixels in HSV (hue, 
saturation, and intensity) color space. The color image 
exhibits a finite range of intensity values and a finite 
range of saturation values. The image is enhanced by 

30 applying a first transformation function to the intensity 
components of the image and a second transformation 
function to the saturation components of the image. The 
first transformation function transforms the range of in- 
tensity values in the image to correspond to the entire 

•35 dynamic range of intensity values available in the video 
processing system. The second transformation function 
transforms the range of saturation values in the image 
to correspond to the entire dynamic range of saturation 
values available in the video processing system. The 

40 transformation functions provide an enhanced image 
having fully optimized intensity and saturation compo- 
nents. The enhanced image is provided by optimally al- 
locating the available dynamic range of intensity and 
saturation values of the video processing system. Sat- 

45 uration and intensity are enhanced to their extremes 
without the introduction of color distortion. 

Brief Description of the Drawings 

50 The file of this patent contains at least one drawing 
executed in color. Copies of this patent with color draw- 
ing^) will be provided by the Patent and Trademark Of- 
fice upon request and payment of the necessary fee. 
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FIG. 1 is a graphical representation of a transforma- 
tion function for mapping the intensity components 
of a color image; 

FIG. 2 illustrates intensity components lor an exem- 
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plary color image comprised of a plurality of pixels: 
FIG. 3 is a histogram showing ihe lelative distribu- 
tion of intensity components for the image of FIG. 2: 
FIG. 4 is a color pictorial representation showing the 
application of the color image enhancement proce- 
dures disclosed herein to an illustrative image; 
FIG. 5 is a pictorial representation showing the ap- 
plication of the gray-scale image enhancement pro- 
cedures disclosed herein to an illustrative optically- 
scanned document: 

FIG. 6 is a graphical representation of a transforma- 
tion function for mapping the Cr chroma component 
of a color image: and 

FIG. 7 is a graphical representation of a transforma- 
tion function for mapping the Cb chroma component 
of a color image. 

Detailed Description of the Preferred Embodiments 

The image enhancement methods disclosed herein 
are directed to improving the appearance of images as 
perceived by the human eye, and/or to render these im- 
ages more suitable for computer analysis. A first pre- 
ferred embodiment enhances color images without de- 
grading image fidelity, and a second preferred embodi- 
ment enhances scanned gray-level document images 
by decreasing the blurring of the image and by increas- 
ing the contrast between light and dark regions of the 
image. Presently-existing video processing systems 
can perform the image enhancement methods dis- 
closed herein in real time. Suitable video processing 
systems may include, for example, a microprocessor 
and a video memory, both of which are present in con- 
ventional video telephones such as the AT&T Picasso 
Still-Image Phone. The AT&T Picasso Still Image Phone 
is disclosed in copending U. S. Patent application Serial 
Number 08/202608. 

With respect to the first embodiment, a color image 
can be enhanced much more effectively in HSV (hue, 
saturation, and intensity) color space than in RGB (red, 
green, and blue) color space. In HSV color space, the 
color information (hue) is isolated and removed from the 
remaining image components (saturation and intensity). 
In this manner saturation and intensity may be en- 
hanced to the maximum extent possible, without intro- 
ducing distortion into the color (hue) component. Any 
color image may be represented by a plurality of pixels 
in HSV (hue, saturation, and intensity) color space. For 
example, well-known methods may be used to convert 
a conventional NTSC, SECAM, RGB, or PAL video sig- 
nal into HSV color space. In some cases, this conver- 
sion process is implemented by performing relatively 
simple computations, but in other cases, nonlinear 
mathematical operations are required. Such nonlinear 
operations are computationally expensive. Therefore, 
the immediately following description will consider im- 
age enhancement in the context of HSV color space, 
after which image enhancement in other color spaces 



will be discussed. * 

Every color image exhibits a finite range of intensity 
values and a finite range of saturation values. Moreover, 
in HSV space, these intensity and saturation values are 

5 specified as individual components of the image. Con- 
ventional video processing systems are equipped to 
process a finite range of intensity values, as well as a 
finite range of saturation values. However, the range of 
intensity values present in the image is not necessarily 

10 identical to the total dynamic range of intensity values 
which conventional video processing systems are 
equipped to process. Similarly, the range of saturation 
values present in the image is not necessarily identical 
to the total dynamic range of saturation values which 

*5 conventional video processing systems are equipped to 
process. 

Typically, the available dynamic range of the video 
processing system is much wider than the range of in- 
tensity and saturation values included in a color image. 

20 Therefore, unless image enhancement methods are ap- 
plied to the image, only a portion of the available dynam- 
ic range of the video processing system will be utilized. 
If the image is underexposed, the intensity values of the 
image are clustered towards one extreme of the availa- 

25 ble dynamic range of the video processing system, and 
remaining portions of the dynamic range are relatively 
underutilized. If the image is overexposed, the intensity 
values of the image are clustered towards another ex- 
treme of the available dynamic range, and remaining 

30 portions of the dynamic range are likewise underutilized. 
A first transformation function is applied to the in- 
tensity values of the image to optimally utilize the avail- 
able intensity value dynamic range of the video process- 
ing system. A second transformation function is applied 

35 to the saturation values of the image to optimally utilize 
the available saturation value dynamic range of the vid- 
eo processing system. The hue values of the original 
image are not changed. The purpose of the first trans- 
formation function is to allocate the dynamic range of 

<o intensity values provided by the video processing sys- 
tem to optimally correspond to the dynamic range of in- 
tensity values included in the original color image. The 
second transformation function allocates the dynamic 
range of saturation values provided by the video 

^5 processing system to optimally correspond to the dy- 
namic range of saturation values included in the original 
color image. 

The first transformation function is a piece wise lin- 
ear mapping function in two dimensions having three 

50 substantially linear regions defined by four intensity val- 
ue points. A first dimension represents intensity input 
values (i.e., intensity values included in an image to be 
processed), and a second dimension represents inten- 
sity output values (i.e., optimized intensity values for the 

55 enhanced image). A first intensity value point is repre- 
sented by the minimum intensity value in a given image, 
a second intensity value point is represented by the 
maximum intensity value in this image, and the third and 
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fourth intensity value points are selected to be between 
the first and second intensity value points. Selection of 
specific values for the third and fourth intensity value 
points are determined by the extent to which a given im- 
age is under- or over-exposed. 5 

Refer now to FIG. 1 . which is a graphical represen- 
tation of an illustrative transformation function for map- 
ping the intensity components of a color image. Image 
enhancement is performed by transforming the intensity 
values of the image using a piece wise linear mapping w 
function 102 for the intensity component The piece wise 
linear mapping function 102 consists of three line seg- 
ments 104, 106, 108. Line segment 104 is defined by 
points vmin and vlower, line segment 106 is defined by 
points vlower and vupper and line segment 108 is de- is 
fined by points vupper and vmax vmin and vmax de- 
note the minimum and maximum intensity levels, re- 
spectively, in the original image. vlower and vupper de- 
termine the dynamic range allocation tor lower interme- 
diate, and higher ranges of intensity levels. 20 

The range of intensity values in the original image 
specifies values for vmin and vmax. The values of 
vlower and vupper are then selected to optimize image 
enhancement for a given image, based upon the char- 
acteristics of the image to be enhanced. For example, 25 
if the original image to be enhanced is under-exposed, 
Woiverand vupper should be selected with a view to- 
wards enhancing lower intensity levels. Therefore, a 
greater proportion of the available dynamic range of the 
video processing system should be allocated to the low- 30 
er intensity range. Such an allocation is accomplished 
by selecting vlower to be relatively close to vmin, and 
by selecting vupperXo be relatively far away from vmax. 
In this manner, high intensity levels are compressed, 
and low intensity levels are expanded. 35 

If the original image is over-exposed, vupper 
should be relatively close to vmax, and vlower should 
be relatively far away from vmin In this manner lower 
intensity levels are compressed, and high intensity lev- 
els are expanded. A greater portion of the video 40 
processing system's dynamic range will then be allocat- 
ed to higher intensity levels^ 

FIG. 1 includes an identity mapping function 103 
which will be discussed hereinafter in connection with 
the second transformation function and image satura- 45 
tion values. 

With reference to FIG. 2 and 3, specific calculations 
for vupperand vlowermay be performed by developing 
an intensity value histogram for the image to be en- 
hanced. Such an histogram divides the range of inten- so 
sity values present in a given image, i.e., the range from 
vmin and vmax, into a plurality of sub-ranges. For each 
of the plurality of sub-ranges, the number of pixels in the 
image having intensity values in this sub-range is deter- 
mined. The first moment of the histogram may be con- ss 
ceptualized as the "center of gravity" of the histogram. 
The "center of gravity, i.e. the first moment, may be cal- 
culated by using the formula 



First 
Moment 
of 

Histogram 



N 



_ 10/ 



A/ 
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where Ij represents a specific intensity level, h { is the 
number of pixels having intensity level Ij, and N is the 
total number of intensity levels (sub-ranges). For exam- 
ple, intensity level 301 (FIG. 3) may be conceptualized 
as representing intensity level one (I,), intensity level 
303 may be conceptualized as representing intensity 
level two (l 2 ), and intensity level 319 may be conceptu- 
alized as representing intensity level ten (l 1 0 ). In this ex- 
ample, there are ten intensity levels (i.e., ten sub-rang- 
es), so N is equal to 10. 

FIG. 2 illustrates intensity components for an exem- 
plary color image 207 comprised of a plurality of pixels 
201 , 203, and 205. Each pixel 201 , 203, 205 is associ- 
ated with a specific value representing the intensity 
component for that pixel. For example, pixel 201 is 
white, which may be defined as representing a relatively 
low value of intensity. In this manner pixel 205, in black, 
represents a relatively high value of intensity, and pixel 
203, in diagonal cross-hatching, represents an interme- 
diate value of intensity with respect to pixels 201 and 
205. 

FIG. 3 is an intensity value histogram showing the 
relative distribution of intensity components for the color 
image 207 of FIG. 2. The intensity value histogram 
shows the number of pixels in a given image that have 
specific intensity values. The y-axis : labelled in "number 
of occurrences", is in units of pixels, and thex-axis rep- 
resents intensity levels. In the example of FIGs. 2 and 
3, ten different intensity values are employed to repre- 
sent the pixel intensity values of the image. With refer- 
ence to FIG. 3, the lowest intensity value is represented 
by intensity value 301 (shown in white), intermediate in- 
tensity values are represented by intensity values 303, 
305, 307. 309, 311, 313, 315, and 317, and the highest 
intensity value is represented by intensity value 319 
(shown in black). From the intensity value histogram of 
FIG. 3, it is observed that the color image 207 includes 
48 pixels having intensity value 301, 14 pixels having 
intensity value 303, 22 pixels having intensity value 305, 
etc. The first moment of the intensity histogram is the 
intensity value including the greatest number of pixels 
which, in the present example, is intensity value 301 
having a total of 48 pixels. 

The first moment of the intensity value histogram 
provides an indication as to the relative over- or under- 
exposure of a given image. The first moment is closer 
to vmin (FIG. 1 ) for an underexposed image, and closer 
to vmaxior an overexposed image. Based upon the first 
moment of the intensity value histogram (FIG. 3), two 
intensity levels, vlower and vupper, are calculated 
such that, as the first moment becomes increasingly 
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closer lo vmin. vupper increases and v\o wer decreas- 
es. As the first moment becomes increasingly closei lo 
vmax. vupper decreases and vlower increases. 

Let the full dynamic range of the video processing 
system be given by (VMIN, VMAX], where VMIN and 
VMAX correspond to the minimum and maximum inten- 
sity values capable of being represented in a given video 
processing system. In the case where a video process- 
ing system consists of a video display device, VMIN and 
VMAX are selected to correspond to the minimum and 
maximum intensity values capable of being displayed. 
Intensity values between vmin and Waiver are mapped 
linearly to the range [VMIN, VLOWER] , where VLOW- 
ER = VMIN + {{vlower - vmln)/2). Intensity values be- 
tween vlower and vupper are linearly mapped to the 
range of [VLOWER, VUPPER], where VUPPER = 
VMAX - {{vmax - vupper)/3}. Intensity values in the 
range [vupper, vmax] are linearly mapped to the range 
[VUPPER, VMAX]. Note that the denominators of the 
equations for VUPPER and VLOWER include a value 
of 3 This value is shown for illustrative purposes. Values 
differing somewhat from 3 may be employed such as, 
lor example, 2.5 or 3.7. Moreover, a different value (i.e., 
4 0 1 may be employed in the denominator of VUPPER, 
ic^ihve to the value employed in the denominator of 
V.OWER (i.e.. 2.3). 

The mapping of the intensity values as described in 
I'-.e ^mediately preceding paragraphs permits proper 
rthucrthon of the dynamic range of a given video 
processing system to the actual range of intensity val- 
ues present in an image to be enhanced. 

A second transformation function is used to trans- 
form ihc saturation components of the original image in- 
to saturation components for the enhanced image. This 
second transformation function is determined by apply- 
ing h histogram stretchin g procedure to a saturation his- 
togram representing the saturation components of the 
original image. A first saturation value point, smin, rep- 
resents the minimum saturation value in the image, and 
a second saturation value point, smax, represents the 
maximum saturation value in the image. 
•y An upper limit of saturation, supper, is defined as 

j 

that saturation level where the area under the saturation 
histogram from that level to the maximum level of satu- 
ration smax, is equal to a small percentage of the total 
number of pixels in the image. For example, a suitable 
percentage could be approximately 0.5% of the total 
number of pixels. Saturation components in the range 
of smin to supper are linearly mapped to the entire dy- 
namic range of saturation values [SMIN, SMAX] availa- 
ble in the video processing system. In this manner, small 
perturbations in the saturation values of pixels are ig- 
nored without compromising the integrity of the en- 
hanced image. 

Using the first and second transformation functions, 
the intensity and saturation values of the original image 
are modified. The hue components of the image are not 
changed, and remain the same from the original image 



lo the enhanced image. The intensity values, as well as 
the color saturation values, of the original image have 
been enhanced to the maximum extent practicable for 
use in the operational environment of an arbitrarily-se- 
5 lected video processing system. The result is an opti- 
mally enhanced color image without the introduction of 
color distortion. 

In some system applications, it may not be feasible 
to determine the available dynamic range of the video 
io processing system which will be used to perform image 
enhancement For example, the dynamic range specifi- 
cations of the video processing system may be un- 
known, or it may be desirable to perform image en- 
hancement using any of a plurality of arbitrarily-selected 
is video processing systems, each of which has different 
characteristics. In such applications, a different set of 
first and second transformation functions could be em- 
ployed for each video processing system, wherein each 
set of transformation functions is optimized for a specific 
20 video processing system. Alternatively, one set of arbi- 
trarily-determined transformation functions may be em- 
ployed, wherein the functions are selected to provide 
useful transformations for a wide variety of video 
processing systems, even though the functions are not 
25 fully optimized for use with any one system. 

The techniques described in the preceding para- 
graphs with reference to the first and second transfor- 
mation functions may be applied to a color space other 
than HSV, although such an application will not neces- 
30 sarily yield optimum results. The process of converting 
other color spaces into the HSV domain is often com- 
putationally intensive, placing excessive demands on 
system hardware. For some system applications, such 
as video telephones which communicate using NTSC 
35 video signals, the required computations would man- 
date the use of undesirably expensive hardware. 

The process of converting a non-HSV color space 
into the HSV domain introduces other problems in ad- 
dition to computational complexities. Certain colors in 
*o one color space may not be physically realizable in an- 
other color space. For example, some colors in the YCb- 
Cr color space of conventional NTSC video signals may 
convert into negative values of color in RGB color space. 
Such negative values are not physically realizable. 

In view of the foregoing difficulties involved in con- 
verting a non-HSV color space into the HSV domain, a 
method is disclosed for enhancing a color image directly 
in conventional NTSC YCbCr color space. This method 
operates in real time and is applicable to any video de- 
50 vice which captures color images in an NTSC format. 

The luminance (Y) component of a conventional 
NTSC signal is enhanced following the procedures de- 
scribed above in connection with the intensity compo- 
nent of the HSV color image. Then, if the chrominance 
55 components Cb and Cr are modified by the same 
amount as the luminance (Y) component, color satura- 
tion will remain the same as in the original image. The 
fact that the color saturation will remain the same can 
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be verified through mathematical derivations known to 
those skilled in the art and disclosed in a textbook enti- 
tled, Computer Graphics, Principles and Practice, by J. 
Foley et aL published by Addtson-Wesley in 1991 . How- 
ever, in order to optimally enhance a color image ! the s 
saturation components of the image can be further mod- 
ified to improve the overall saturation of the image. 

The saturation components of the NTSC image are 
enhanced by modifying the chrominance components, 
Cb and Cr, by a greater amount than the amount by 10 
which the luminance component, Y, has been en- 
hanced. In other words, the slope of the transformation 
function applied to the chrominance components (the 
chrominance transformation function) should be greater 
than the slope of the transformation function applied to is 
the luminance component (the luminance transfer func- 
tion). In cases where the luminance component is mod- 
ified to a relatively large extent by the luminance trans- 
formation function, this is an indication that the original 
image is relatively unsaturated. The modification per- 20 
formed by the luminance transformation function is in- 
versely proportional to the amount of saturation in the 
original image. Therefore, in cases where the saturation 
is relatively low, the saturation should be increased pro- 
portionally, with reference to circumstances where the 2$ 
saturation is relatively high. 

One technique for measuring the amount by which 
saturation should be increased is by computing the de- 
viation from linearity of the luminance transformation 
function. With reference to FIG. 1 , the y axis which rep- 30 
resented "intensity output" is now conceptualized as 
representing luminance output levels, and the x axis is 
now conceptualized as representing luminance input 
levels. For each of a plurality of luminance levels, the 
absolute difference between the piece wise linear map- 35 
ping function 102 and an identity mapping function 103 
is calculated. The absolute differences for all of the lu- 
minance levels are then summed up to obtain the total 
deviation from linearity, denoted by Dev. Then, the fol- 
lowing equation is employed to calculate the value of ao 
incr. incr ls used to determine the amount by which the 
chrominance components should be increased: 1.0 < 
(incr= Oev/8000) < 2.0. The value of 8000 in the de- 
nominator is a representative value set forth for purpos- 
es of illustration. Other numerical values may be em- *5 
ployed in the denominator in place of 8000 such as, for 
example, 7000, which also provides useful perform- 
ance. 

Refer now to FIG. 6, which is a graphical represen- 
tation of a transformation function for mapping the Cr so 
chroma component of a color image, and also to FIG. 
7, which is a graphical representation of a transforma- 
tion function for mapping the Cb chroma component of 
a color image. The slope of the piecewise linear map- 
ping function 102 (FIGs. 6 and 7) for the range [vlower, 55 
vupper] (FIG. 1 ) is multiplied by incrXo obtain the slope 
of the chrominance transfer function. The actual mini- 
mum and maximum levels of chrominance in the original 



image are mapped to the efitire chrominance compo- 
nent dynamic range (FIGs. 6 and 7). A white point 105 
corresponding to a chrominance level of 128 for both Cr 
(FIG. 6) and Cb (FIG. 7) is mapped to itself by the 
chrominance transfer function. Using the luminance and 
chrominance transfer functions, the luminance and 
chrominance components values of the original image 
are transformed to create an optimally enhanced image 
directly in YCbCr color space. YCbCr color space may 
be employed in the context of conventional PAL, SE- 
CAM, and/or NTSC video signals. 

Another preferred embodiment disclosed herein 
enhances gray-level textual and/or documeni images by 
decreasing the blurring of the image and by increasing 
the contrast between light and dark regions of the im- 
age. These gray-level images may be represented, for 
example, in NTSC YCbCr color space. A major objective 
of text image enhancement is to increase the readability 
of text. Since readability of a text image is determined 
by the luminance component of the image, this method 
focuses on enhancing the luminance (Y) component. If 
the color information present in a given image is very 
limited, then the chrominance components Cb and Cr 
may not require any processing. Otherwise, Cb and Cr 
may be processed as described in the immediately pre- 
ceding paragraphs. 

Detection of the amount of color information in an 
image is performed by determining the amount of histo- 
gram spread of the Cb and Cr components. If the histo- 
gram spread is smaller than a given threshold, i.e., ap- 
proximately 23, then the text image is treated as a gray- 
level image and only the luminance component is en- 
hanced. 

One possible approach for textual image enhance- 
ment is to stretch the Y histogram to the maximum avail- 
able dynamic range, thereby enhancing image contrast. 
Although this approach improves the image, further im- 
provements are possible. Pursuant to the method dis- 
closed herein, the background gray level of the text im- 
age is determined. Textual images generally contain a 
relatively limited number of discrete gray levels, as is 
the case for binary documents, black text on white back- 
ground, etc. The background gray level corresponds to 
a significant peak in the Y (luminance) histogram. Such 
a luminance histogram is similar to the intensity value 
histogram, except that pixel luminance values are 
graphed in lieu of pixel intensity values. The peak in the 
Y (luminance) histogram is detected by checking the 
gray level at which the peak detection signal crosses 
over zero from negative values to positive values. The 
peak detection signal is generated by computing the dif- 
ference between the cumulative histogram and a 
smoothed version of the cumulative histogram, the de- 
gree of smoothing determining the sensitivity of the peak 
detection. 

For purposes of the present example, the one peak 
in the Y histogram which represents the background 
gray level must be detected. This detection process is 
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dependent upon whether the original document (i.e.. 
textual image) has a high gray level backgiound 01 a low 
gray level background. For high gray level backgrounds, 
starting from the maximum gray level in the histograms 
the first peak encountered in the histogram is the back- s 
ground gray level. For low gray level backgrounds : start- 
ing from the minimum gray level in the histogram, the 
first peak encountered in the histogram is the back- 
ground gray level. 

Assume, for purposes of the present example, that J0 
the textual image has a high gray level background de- 
noted as ybkg. As stated above, an alternative for lo- 
cating the background gray level is merely to locate the 
first peak from the high gray level end of the Y histogram. 
However, the Y histogram should be smoothed first (us- is 
ing conventional smoothing procedures); otherwise the 
searching process will locate a peak in a local area hav- 
ing a relatively small perturbation, while entirely missing 
the larger peak representing the actual background gray 
level. 20 

The minimum foreground value of the document, 
denoted as ymin, can be located by examining the Y 
histogram from its lower gray level end. During the 
search, the first few minimum values corresponding to 
a small percentage of total number of pixels are ignored. 25 
This small percentage may be in the vicinity of 0.1%, as 
an illustrative example, thereby allowing the method to 
skip relatively small perturbations in the observed gray 
levels of the textual image. 

The contrast of the textual image is enhanced by 30 
mapping the Y (luminance) component from [ymin, yb- 
kg] to [YMIN, YMAX] , where YMIN and YMAX are : re- 
spectively, the minimum and maximum Y values of the 
video processing system. Before mapping the Y values, 
a technique generally known to those skilled in the art 35 
as unsharp masking is applied to the Y components of ' 
the entire image to remove any blurring from the textual 
image. Conventionally, this sharpening operation caus- 
es the appearance of halos around the text edges, which 
is not very pleasing to human eyes. Since these halos *o 
are caused by overshoots in pixel values above the 
background gray level, ybk& the halos can be easily 
removed using standard methods known to those skilled 
in the art after mapping the Y values from {ymin, ybkg] 
to [YMIN, YMAX]. Note that the mapping function should *s 
be determined before the unsharp masking is applied to 
the image: otherwise, the background level cannot be 
detected correctly and the halos can no longer be re- 
moved. 

Due to the fact that the textual image enhancement so 
method described herein takes into consideration the 
characteristics of the actual textual image, the method 
effectively removes blurring from the image and maxi- 
mizes contrast. In addition, if the original and the en- 
hanced images are both compressed in a format known ss 
to those skilled in the art as JPEG, relatively few block- 
ing artifacts will appear in the enhanced image. Fewer 
blocking artifacts appear because the enhanced image 
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includes a more uniform bacliground and more uniform 
foreground gray levels. Although the textual image en- 
hancement method was described in the context of 
YCbCr color space, the method is also applicable to oth- 
er color spaces. 

FIG. 4 is a pictorial representation showing the ap- 
plication of the color image enhancement procedures 
disclosed herein to an illustrative color image. In order 
to demonstrate the effectiveness of the methods dis- 
closed herein, an illustrative image 400 was manually 
degraded to provide a test image 402. Illustrative image 
400 and test image 402 are in YCbCr color space. First 
enhanced image 404 is the enhanced image which re- 
sults when a transfer lunction is applied only to the Y 
component of the test image 402. As was predicted 
above, the color of the enhanced image 404 is not en- 
hanced relative to the test image 402. 

Second enhanced image 406 is the image which 
results when transfer functions are applied to the lumi- 
nance and chrominance components of the test image 
402. Color distortion, caused by interactions between 
the various color components, is clearly observed. 
Some type of color compensation scheme must be used 
to account for interactions between the various color 
components, and to remove such color distortion. 

Third enhanced image 408 is the image which re- 
sults when any of a number of commercially-available 
image processing software packages are utilized. 
These software packages generally includes some type 
of color compensation scheme which accounts for inter- 
actions between the various color components. Even 
though the color is not distorted, these prior art software 
packages do not provide an optimally enhanced image. 

The test image 402 may be optimally enhanced by 
using the methods disclosed herein, as described above 
in connection with FIGs. 1-3. Such an optimally en- 
hanced image 410 is shown in FIG. 4. By comparing 
each of the enhanced images 404, 406, 408, 410 with 
the illustrative image 400, it is observed that the meth- 
ods disclosed herein enhance contrast and color satu- 
ration to the maximum extent practicable. Moreover, the 
methods disclosed herein do not provide color distor- 
tion. 

FIG. 5 is a pictorial representation showing the ap- 
plication of the gray-scale image enhancement proce- 
dures disclosed herein to an illustrative optically- 
scanned document. Textual image 500 is a portion of an 
original textual image obtained from an AT&T Picasso 
Still Image Phone. The AT&T Picasso Still Image Phone 
is disclosed in copending U.S. Patent application Serial 
Number 08/202608. Enhanced image 502 was pre- 
pared using a Y histogram stretching process as de- 
scribed above, followed by unsharp masking. It is ob- 
served that the contrast is not optimized, and that halos 
have been introduced by the masking process. En- 
hanced image 504 was prepared using the mapping 
function described above and generated from the de- 
tected background gray, level. Note that some blurring 
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is present. Enhanced image 506 was prepared using the 
methods disclosed herein. It is observed that the con- 
trast ol the image has been optimized, and the blurring 
has been removed. 



Claims 



1. 



3. 



A method of enhancing a color image using a video 
processing system equipped to represent a dynam- 10 
ic range of intensity values and a dynamic range of 
saturation values, the method comprising the fol- 
lowing steps: 

representing the color image by a plurality of is 
pixels in HSV (hue, saturation, and intensity) 
color space, the color image having a finite 
range of intensity values and a finite range of 
saturation values; 

applying a first transformation function to the fi- 20 
nite range of intensity values and a second 
transformation function to the finite range of 
saturation values; the first transformation func- 
tion transforming the first finite range of inten- 
sity values to correspond to the entire dynamic 25 
range of intensity values; the second transfor- 
mation function transforming the finite range of 
saturation values to correspond to the entire dy- 
namic range of saturation values; 
the first and second transformation functions 30 
thereby providing an enhanced image having 
fully optimized intensity and saturation values. 

A method of enhancing a color image as set forth 
in Claim 1 wherein the first transformation function 35 
is a piece wise linear mapping function in two di- 
mensions having three substantially linear regions 
defined by four intensity value points; the first di- 
mension representing the finite range of intensity 
values, and the second dimension representing the 40 
dynamic range of intensity values, 

the method further comprising the steps of: 
defining a first intensity value point as being 
represented by the minimum intensity value in 45 
the image; 

defining a second intensity value point as being 
represented by the maximum intensity value in 
the image; 

and selecting third and fourth intensity value so 
points having values in the first dimension be- 
tween the first and second intensity value 
points. 

A method of enhancing a color image as set forth 55 
in Claim 2 wherein the piece wise linear mapping 
function consists of first, second, and third line seg- 
ments, the first line segment being defined by points 



vmin and vlower, the second line segment being 
defined by points vlower and vupper, and the third 
line segment being defined by points vupper and 
vmax, such that vmin is the minimum intensity val- 
ue in the image, vmax is the maximum intensity val- 
ue in the image: 

the method further comprising the step of: 
selecting vlower and vupper based upon the 
relative overexposure or underexposure of the 
image such that: 

if the image is relatively under-exposed, vlower 
is selected to be relatively close to vmin and 
vupper is selected to be relatively distant from 
vmax, thereby compressing high intensity val- 
ues and expanding low intensity levels; and 
if the image is relatively over-exposed, vupper 
is selected to be relatively close to vmax, and 
vlower is selected to be relatively distant from 
vmin, thereby compressing lower intensity val- 
ues and expanding higher intensity values. 

A method of enhancing a color image as set forth 
in Claim 3 wherein vupper and vlower are calcu- 
lated by performing the steps of: 

developing an intensity value histogram for the 
image to be enhanced by dividing the range 
from vmin and vmax into a plurality of sub- 
ranges such that, for each of the plurality of 
sub-ranges, the number of pixels in the image 
having intensity values in this sub-range is cal- 
culated; the center of gravity of the intensity val- 
ue histogram being defined as the first momont 
and being indicative as to the relative over- or 
under-exposure of the image, such that the first 
moment is relatively closer to vmin than to 
vmax for an underexposed image and relative- 
ly closer to vmax than to vmin for an overex- 
posed image; and 

selecting vlower and vupper such that, as the 
first moment becomes increasingly closer to 
vmin, vupper increases and vlower decreas- 
es, and, as the first moment becomes increas- 
ingly closer to vmax, vupper decreases and 
vlower increases. 

A method of enhancing a color image as set forth 
in Claim 4 wherein vupper and vlower ate calcu- 
lated by performing the steps of: 

defining the full dynamic range of intensity val- 
ues of the video processing system as [VMIN, 
VMAX], wherein VMIN corresponds substan- 
tially to the minimum intensity value capable of 
being represented in the video processing sys- 
tem, and VMAX corresponds substantially to 
the maximum intensity value capable of being 
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represented in the video processing system; 
linearly mapping intensity values in the range 
of vmin and vlower in the first dimension to in- 
tensity values in the range of VMIN and VLOW- 
ER in the second dimension, where VLOWER 5 
= VMIN + [{vlower - vmin )/m], where m is a 
real number in the approximate range of 2 to 4: 
linearly mapping intensity values in the range 
of Woiverand vupper in the first dimension to 
intensity values in the range of VLOWER and 10 
VUPPER in the second dimension, where 
VUPPER = VMAX-{(wnax- vupperyk), where 
k is a real number in the approximate range of 
2 to 4: and 

linearly mapping intensity values in the range is 
of vupper and vmax in the first dimension to 
the intensity values in the range of VUPPER 
and VMAX in the second dimension. 

6. A method of enhancing a color image as set forth 20 
in Claim 5 wherein the video processing system 
consists of a video display device and VMIN and 
VMAX are selected to correspond to the minimum 
and maximum intensity values capable of beingdis- 
played on the video display device. 25 

7. A method ol enhancing a color image as set forth 
in Claim 5 wherein a second transformation function 
is used to transform the saturation components of 

the original image into saturation components for so 
the enhanced image 



12. A method of enhancing a*color image as set forth 
in Claim 10 wherein saturation components in the 
range of smin\o supper are linearly mapped to the 
entire dynamic range of saturation values [SMIN. 
SMAX] available in the video processing system : 
whereby small perturbations in the saturation val- 
ues of pixels are ignored without compromising the 
integrity of the enhanced image. 

13. A method of enhancing a color image as set forth 
in Claim 10 wherein the intensity values of the im- 
age are modified using the first transformation func- 
tion to provide enhanced image intensity values, 
and the saturation values of the image are modified 
using the second transformation function to provide 
enhanced image saturation values. 

14. A method of enhancing a color image as set forth 
in Claim 1 3 wherein the hue components of the im- 
age to be enhanced constitute the hue components 
of the enhanced image. 



6. A method of enhancing a color image as set forth 
in Claim 7 wherein the second transformation func- 
tion is determined by applying a histogram stretch- 3S 
ing procedure to a saturation histogram represent- 
ing the saturation components of the color image. 

9. A method of enhancing a color image as set forth 

in Claim 8 wherein the histogram stretching proce- 40 
dure includes the steps of selecting a first saturation 
value point, srnin, representing the minimum satu- 
ration value in the image, and a second saturation V 
value point, smax, representing the maximum sat- 
uration value in the image 45 

10. A method of enhancing a color image as set forth 
in Claim 9 wherein an upper limit of saturation, sup- 
per, is defined as that saturation level where the ar- 
ea under the saturation histogram from that level to 50 
the maximum level of saturation, smax, is equal to 

a small percentage of the total number of pixels in 
the image. 

11. A method of enhancing a color image as set forth 55 
in Claim 10 wherein the small percentage is approx- 
imately equal to 0.5% of the total number of pixels 

in the image. 
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Real-time image enhancement techniques 



Improved image enhancement methods are 
disclosed which may be performed on conventional vid- 
eo processing systems. These methods operate in real 
time to improve the appearance of images as perceived 
by the human eye, and/or to render these images more 



suitable for computer analysis. A first preferred embod- 
iment enhances color images without degrading the fi- 
delity of the image. A second preferred embodiment en- 
hances scanned gray-level document images by de- 
creasing the blurring of the image and increasing the 
contrast between light and dark regions of the image. 
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